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The Communication Program of the Far West Laboratory for Educational 
Research and Development is developing a self-contained training package 
on Ir .ructional Planning for administrators , teachers, and other groups 
involved in school planning. The training package consists of three 
sequential units: Problem Analysis, Goal Setting, and Objectives and 

Evaluation. Preliminary field test results, showing that the preliminary 
form of the units partially achieve their objectives of improving trainees 1 
attitudes, knowledge, and skills relevant to instructional planning, are 
being used to revise the training units prior to further testing. The 
Problem Analysis unit is discussed in depth to illustrate the research and 
development cycle used to develop the training package. 
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Introduction 



An important move in education today is the effort to make local school 
districts accountable . Schools are seeking to evaluate the success of school 
programs in terms of their impact on student learning. They are also being 
called upon to clarify the objectives of the school program, and to demonstrate 
to concerned groups that these objectives are relevant to the goals and problems 
of society. 

This emphasis on accountability and relevance nas begun to influence 
ecucators' views concerning the type of training needed by the school people 
who plan, select, manage, and evaluate instructional programs. A series of 
"training packages" is being developed by the Far West Laboratory for Educa- 
tional Research and Development in order to meet some training needs 
in the area of instructional planning and management. This paper describes 
the preliminary development of the Laboratory's first training package, 
on Instructional Planning. Even though this training package is not yet 
ready for widespread dissemination, it should be of interest to educators 
concerned about new techniques for training school administrators and other 
school people. In this paper, therefore, the developer: will describe the 
package and explain the procedures being used in Us development. 

Obje c tives 

When development of the first training package was undertaken by the Labora- 
tory's Communication Program, the Program staff decided to divide the package 
into three sequential training units: Problem Analysis , Goal Setting and 

Objectives and Evaluation. The initial description of each unit included a 
list of the overall goal: which it was to achieve. The Problem Analysis Uni t 



aims to train school people to define problems in terms of existing and 



desired student outcomes based upon a variety of problem "signals," 
to determine what information is needed to analyze each problem and 
how to obtain it, and to use the information to judge tne validity 
and relative seriousness of each problem. The Goal Setting Unit seeks 
to improve their ability to relate validated problems to school goals, 
to determine the potential impact on the school system of solving 
one or more problems, and to develop goals and goal indicators which 
indicate in broad, but behavioral, terms the desired student oitcomes 
in high priority problem areas. The Obj ectiv es and Evaluation Unit 
aims to improve school people's ability to break down the goal indicators 
for given problem areas into more specific student outcome objectives, 
to analyze objectives for their appropriateness in various settings, 
and to develop an evaluation plan to measure the achievement of objectives 
following instruction. 

Actual development of the training units began with the specification 
of performance objectives for school personnel after they complete each unit. 
The overall goals were broken down into performance objectives of three 
types: knowledge objectives (concerned with increasing trainees’ knowledge 

of the processes involved in instructional planning); skill objectives 
(concerned with increasing trainees' ability to perform these processes); 
and affective objectives (concerned witn improving trainees' attitudes 
toward, or desire to apply, these processes in their work). The training 
units will become available to school districts only after the developers 
have demonstrated through field use that they successfully achieve their 
performance objectives. 
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A few examples of the performance objectives for Unit 1, Problem 
Analysis, illustrate what trainees are to learn by using the training 
package. Standards for the skill objectives are presently being established. 

Knowledge : Given the names of the three phases of Problem Analysis 

(Defining~$tudent Outcome Problems, Planning to Collect Information to 
Analyze Student Outcome Problems, and Determining Validity and Seriousness 
of Student Outcome Problems) each trainee will be able to describe in 
writing the major objectives of each phase at a level judged by the de- 
veloper to be "5" or better on a 7-po 4 nt scale going from very poor ("l") 
to excel lent ( "7 M ) . 

Skill : Given a list of alternative methods for collecting information 

to analyze prob’ems, and a maximum budget limit for collecting the informa- 
tion (e.g., $600.), each trainee will specify an information collection plan 
which includes the most cost-effective methods--that is, those methods which 
will best answer his questions while staying within the budget limit. 

Ski 1 1 : Given a set of simulated problem materials which refer to several 

student outcome problems in the same problem area, each trainee will 
be able to discriminate those student outcome problems which appear to be 
valid, those problems that appear to be invalid, and those problems whose 
validity cannot be determined from the information given. 

Affect ive: Each trainee will rate the usefulness of problem analysis 

for helpTng plan the instructional program to be "5" or better on a 7-point 
scale going from not useful ( M 1 M ) to extremely useful ('7"). 



Product Descr iption 

Our major specifications for the operational (final) form of each training 
unit are that it be: 

a. Basic . Each training unit is meant to provide coverage of only those 
skills that are basic to instructional planning. Thus the "entry level" of 
crainees need not be high, and the skills covered are those that appear to be 
of general relevance to instructional planners having a wide range of duties 
and responsibilities. (Another version of the training package, to be devel- 
oped later, will cover more advanced skills involved in instructional planning.) 
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b. I ndependent . Although the three training units in the package were 
conceptualized as falling into a logical sequence, we are designing the units 
so that they may be taken in ary sequence or each one can be used alone. A 
general orientation to the training package will describe tne processes 
covered in each training unit, and the simulated background information will 
be common to all the units. 

c. Self - contain ed. When the traininy units become operational, they 
will no longer be dependent upon the presence of the developer, will include 
ali the materials and instructions necessary for a Coordinator to conduct 
the course, and will be appropriate for a variety of training settings and 
schedules . 

Since each training unit is made up of various "elements" (e.g. guidelines, 

training exercises) and uses several methods (role playing, team work, etc.), 

the developers also included in their product specifications the condition 

% 

that each element and method be valued, or favorably rated by trainees. 

To develop a product which will meet the objectives and specifications 
described, a systems approach is being used. Thus each training unit repre- 
sents one step in a comprehensive set of procedures for systematical ly attack- 
ing instructional problems, lhe training units require the trainee to engage 
in simulation activities in which he acts as an instructional planner In a district- 
wide Instructional Planning Team, For example, the Problem Analysis training unit 

begins by presenting an assortment of memoranda, news clippings, etc. which signal 
various problems in the hypothetical Mid City Unified School District (MCUSD). 
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From this information, the trainee is asked to identity the major problems, 
not in terms of any personal or nation-wide significance he may attach to 
them, but strictly in terms of the situation that exists in Mid City. The 
content of each training unit is programmed, or sequenced to provide the 
same set of instructional experiences with every presentation. First the 
trainee receives written guidelines that describe step-by-step how to per- 
form a given skill, followed by training exercises, complete with detailed 
instructions and worksheets, that give him an opportunity to practice applying 
the skill. Then he receives written feedback on how well he performed. In 
attempting to apply the instructional planning skills, he operates as a member 
of a team. This enables him to experience how varying perceptions and values 
affect group decision-making. 

The graining materials are intended for use by school administrators and 
other people involved in long-range, district-wide instructional planning. 

This may include persons who are aspiring to district-wide positions, as well 
as persons who presently have responsibilities for the instructional program 
of an entire district. Each unit requires 10-15 hours in training sessions, 
which may be arranged on a distributed or massed schedule, depending on the 
site and the Coordinator 's preference. A Training Coordinator conducts the 
sessions. He is knowledgeable about all the training materials and, in 
addition to giving instructions for each training activity, may clarify 
procedures, answer questions, and give trainees additional feedback on their 
performance. 



The R & D Cycle Used to Develo p the T rainlngjackage 
Before the Ins true tional Planning package becomes available for general 
dissemination, it will complete a rigorous development cycle. The cycle begins 

S3 




8 



5 



with conceptualization, in which the need for and feasibility of the package 
are analyzed, and ends with preparation of a plan for disseminating the 
package and providing fc»" quality control. The ultimate outcome of this cycle 
will be an Instructional Planning package that has been demonstrated as 
satisfactorily meeting product specifications and achieving performance 
objectives. 

Evaluation is conducted at every stage in the cycle, and revision 
of the package follows each stage of evaluation. The number of field test patti- 
cipants, as well as the formality of the data collection end analysis, increases 
at each subsequent stage, and the purpose cf evaluation shifts from gathering 
information indicative of needed revisions of the training package to demon- 
strating the effectiveness of the package as a training device for school 
people. 

To date, the Instructional Planning package has completed three stages 
of a seven-stage R & D cycle, and it is therefore in developmental rather 
than operational form. Development began with the Conce ptualization stage. 

First, evidence was gathered demonstrating the need for an administrative 
training papkage dealing with instructional planning techniques. A literature 
review and interviews with administrators from several local school districts 
revealed that a sizeable proportion of school administrators perceived a need 
for better training, including techniques for long-range planning, for commu- 
nicating to the community the major problems facing the schools, and for 
convincing people of the educational priorities they set. Th n proposed 
solution to the need was derived from a literature review of theoretical 
planning models applicable to educational administration. Models of systems 
analysis, problem solving, decision making, and knowledge utilization were 
all reviewed, as well as some existing training programs based on such models. 

O 
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The research literature on training methods, including simulation and gaming, 
inbasket techniques and programmed instruction, was also consulted, The 
proposed solution that emerged was a self-contained training package using 
simulation as the means to present a system analytic approach to instructional 
pi anni ng. 

For the Problem Analysis Unit, a major portion of the conceptualization 
stage was devoted to developing and getting expert ratings of proposed per- 
formance objectives. An objectives questionnaire was administered to prac- 
ticing administrators from several districts, and to Laboratory personnel. 
Respondents rated each objective for its a) worthwhileness, b) feasibility, 
c) achievabi 1 i ty , and d) measurability. On the basis of the mean ratings, 
the developers were able to eliminate a few objectives and conclude that the 
remaining objectives were reasonable ones to attempt to achieve. Conceptuali- 
zation of the training does not end here, however. Evaluation data 
gathered at each subsequent stage of development are used to modify the con- 
ceptualization and to revise or supplement the objectives. 

Next a few tentative elements of the Problem Analysis unit were developed 
This marked the beginning of the second development stage, Feasibil i ty Testing 
A prototype of the training unit was developed, including a short explanation 
of tne proposed training program, a one-hour written exercise using simulated 
materials, and a short followup discussion on the usefulness of the exercise 
and the materials provided. The Problem Analysis prototype was tested for 
feasibility with two small groups of potential trainees: a graduate education 

class at a local university and a group of building-level and central office 
administrators from a local school district. The trainees (N=15) provided 
both oral and written feedback on the exercise and suggestions for improvement 
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Now the training unit entered the stage of Preli minary Field Testin g. 

A preliminary form of the Problem Analysis unit was developed and used in 
four field tests over a six-month period. Each field test was an official 
college or university course, in which trainees went through all the train- 
ing and evaluation procedures. A total of fifty trainees participated in 
the preliminary field tests for Problem Analysis. About half of them were 
practicing school administrators, primarily building principals. Most of 
the other trainees were teachers, many of whom had supervisory responsibility 
or were soaking administrative credentials. 

The other two units of the Instructional Planning training package are 
following a development cycle similar to that described for the Problem 
Analysis unit. The Goal Setting training unit has, like Problem Analysis, 
completed the Preliminary field Test stage. The Objectives and Evaluation 
unit is in the Conceptualization stage, but is receiving a concentrated de- 
velopment effort so that all three units of the training package will complete 
their R & f) cycle at the same time. 

Highligh ts of the Preli mi nary 
Finding s and Planned Revisio ns 

Analysis of the data collected during the preliminary field tests has been 
completed, and the findings are being used by the developers to revise the 
training units prior to further testing. The results of subsequent field tests 
will be addressed to the issue of how well the product works, i.e. how 
well it meets performance objectives and product specifications. In this 
paper we shall discuss some general conclusions reached on the basis of the 
preliminary findings, and the plans for revision which these findings indicated. 
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Ski ll Objective s 

To date each field test has included special pre- and post-tests to 
determine the extent to which trainees increase their skills in instructional 
planning as a result of using the training unit. While statistical analyses of 
the test results from the first two field tests provided some evidence of 
improvement in terms of simple, countable scores--for example, the number of 
factors that determine the seriousness of student outcome problems which trainees 
could list--the results were not adequate for testing the more complex 
skills stated in the performance objectives (see p. 3). Therefore * for our 
last preliminary field test we revised the pre- and post test to more closely 
approximate the activities trainees go through during training, including the 
provision of partially filled-in worksheets {rather than blank paper) for their 
responses, and the use of "ideal responses" which raters could use in evaluating 
each trainee's response to each test question. In our subsequent field 
tests we will attempt to evaluate the actual output of the training activities, 
that is, the worksheets that individuals or teams complete during training. 
Kn owledge Ob jectives 

Using several "knowledge questionnaires," we sought information con- 
cerning trainees' knowledge of the purpose of problem analysis and of the 
rnajrr process involved in it, of the concepts presented in the training unit, 
and of the likelihood of certain benefits wi ich a school might receive from 
taking a problem analytic approach to Instructional planning. The results were 
promising but uneven. Trainees were not up to standard in terms of their 
knowledge of the purpose of problem analysis and of each major process in- 
volved in it but demonstrated adequate knowledge, in terms of recall, of most 
of the concepts presented (e.g. problem signal, desired state,s tuasnt outcome). 
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Furthermore they perceived more benefits to be likely from taking a problem 
analytic approach than we deemed reasonable, for example helping a school to 
determine what solution should be adopted for a particular problem, or to 
determine the cost of solving a problem. Based on these results, we are 
revising the training unit to provide a more thorough orientation to the 
purposes of problem analysis and the benefits it will and will not provide. 

We will also "program" the guidelines, so that trainees receive a definition 
of each important concept, see it illustrated by examples, and test themselves 
for knowledge of the concept before they proceed to training activities using 
the concept. 

Affect ive Objectives 

A majority of the test participants judged the processes involved in 
problem analysis to be useful in planning, decision making, and problem 
solving. The major suggestion of those trainees who did not rate the usefulness 
of problem analysis as high was that the unit should help trainees apply 
the skills covered in training to their real-life problems. We intend to 
add an "application" element to the unit which will suggest ways in which the 
skills and knowledge learned in training can be applied to trainees' own school 
problems. We also intend to make our rationale for using simulation clearer, 
namely the fact that simulation provides a common background for trdnees so that 
they can work together on problems, and that it is a means of learning 
skills which can be transferred to other problems than those covered in training. 
Product Sped ficatio ns 

Trainees' reactions suggested chat the first major produce specification, 
i.e. that basic skills be covered, is generally being met. Ine skills 
covered do not appear to oe too difficult exc^n** perhaps for persons almost 
completely unfamiliar with educational administration. Moreover, trainees 
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from a variety of positions in the school hierarchy have found the training 
useful, as described below under trainees' reactions to the elements and 
methods used in the training unit. 

The ability of the training units to function independently, which is 
the second major product specification, will be tested during the Performance 
Test stage. At that time we will have added an improved orientation to the 
training package that can be used with any of the three units, and will begin 
using a few different sites for field testing each of them. 

A significant step was taken toward meeting the unit's third major product 
specification, namely, that the unit be "self-contained," with the introduc- 
tion of a non-staff Training Coordinator during the last two preliminary field tests. 
The Coordinators were given written instructions and briefings by the developers 
prior to the training sessions. However, their role was left fairly un- 
structured so that the developers could learn how to best define it by 
observing the personal styles that were adopted by each Coordinator. We be- 
came convinced after observing these preliminary tests that it is necessary 
for the Coordinator, at the minimum, to be responsible for making physical 
arrangements, scheduling training activities, and giving instructions. 

If the unit is to succeed, these duties cannot be neglected in favor of 
other role preferences. However, we also kurned that the Coordinator could 
add to the value of the training if he also provided leadership and feedback 
to the trainees. This additional role will be suggested in future field tests 
as an option, not a requirement, for the Coordinator, since we do not wish 
the success of the training units to depend on the expertise and leadership 
ability of each individual Coordinator. 

From trainees' ratings and open-ended responses on a questionnaire 
administered at the end of each preliminary field test, we were able to 
learn which elements and methods of the training unit were being favorably 
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received and which, in the opinion of the trainees* needed revision. Most 
trainees rated the content of the Problem Analysis ^uid lines as highly 
useful, endorsed the methods of simulation and team v ark used in training, 
and rated the amount of information presented in the simulated documents 
to be appropriate for making the educational decisions called for. Aspects 
of the unit that were not as favorably received were the use of role-playing, 
the format of the guidelines, the amount of written work required, and in- 
sufficient time for reading assignments, group discussions, and the unit as 
a whole. 

We are attempting to improve the ft le-playing by eliminating the single 
"role 1 ' (i.e., the new Assistant Si pc r A ntendent for Instruction in the simulated 
school district) that all trainee were asked to assume evert when functioning 
as members of a team alung with other trainees. The training unit will be 
revised either to introduce various roles which the team members may play 
(e.g. teacher, principal, curriculum coordinator) or to prescribe a general 
role (an instructional planner for MCUSD) for all trainees. As a means of 
responding to trainees' frequent complaints of being rushed and overworked, 
we plan to introduce a more flexible training schedule at the next stage of 
field testing. This would permit the Coordinator to schedule certain training 
activities (e.g., reading of guideline materials and individual written 
assignments) to be performed outside of class, thus allowing more class 
time for team work and class discussions. The guidelines themselves will 
be reorganized to better serve as reference tools during the training 
exercises . 

Plans f or Further D evel opment 

At present we are entering the Performance stage 'stage 4) for the 
Problem Analysis training j/iit. This involves revising the preliminary 
product based cn our findings, field testing the product with 
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trainees representing a wider range of school and regional variations, and 
implementing a more polished evaluation plan to gather evidence of the 
product's worth as a training device* During this stage there is one final 
opportunity to revise the product in order that it better meet its performance 
objectives and product specifications. In the summer and early fall of 1971 
we will complete Operational Field Testing (stage 5) of the training unit. 

This stage involves field testing the product in its operational (final) form, 
with minimal Laboratory involvement. The main purpose of the Operational 
Test stage is to obtain evidence regarding the product's readiness for release 
to potential users (e.g., university departments of education and 
school districts developing staff training programs). 

The target date for completing the Operational Field Test stage of all 
three training units is November, 1971. At that time we will also have 
developed a plan for dissemination (stage 6) and a plan for maintaining quality 
control (stage 7) over the product after its release. 

The training package that we have described is ope of several now 
being developed or planned by the Communication Program of the Far 
Me st Laboratory for Educational Research and Development. It is hoped that 
this paper has given the reader some insight into the nature of the R & D 
model being used in the Program's development work. 
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